Action recognition in video

Automatic action recognition in video has a broad array of applications, from surveillance to interactive video games. Classic algorithms usually use handcrafted descriptors such as SIFT (see [5]) or HOG (see [3]) to compute feature vectors of videos, and have achieved promising results in the past (see [7]). More recently, Quoc Le and Will Zou at the Stanford AI lab have proved that ISA features obtained from unsupervised learning achieve higher performance, while being much faster to engineer that hand-crafted features (their work is not yet published). SFA features have achieved good results in object recognition as well as position and rotation extraction from artificial video signal (see [4]). In this work, we experiment using SFA features for action recognition. 1 Evaluation framework Our pipeline for action recognition follows the pipeline presented in [7]. It first uses a dense detector to split the videos into spatio-temporal chunks. It computes the SFA feature vector of each of those chunks, and then clusters all those feature vectors (from all the videos). Finally, it defines the feature vector of one video to be the vector (hi), where hi = 1 if and only if one chunk in the video is in the i-th cluster. Those feature vectors are then used by a SVM with a χ2 kernel for classification. 2 One layer SFA We first experiment with a one-layer SFA. For all my experiments, we use movies from the Hollywood2 dataset (see [6]) at half the original resolution, ie. around 300 × 150 px. The size of the patches is 10 × 10 px, so the input dimension is 100. A first SFA pass reduces the dimension to 48. Then, quadratic expansion is performed, and finally SFA selects the 32 slowest features, so that the output dimension is 32. Since the temporal size of each spatio-temporal chunk is 10 frames, the dimension of the feature vector of each spatio-temporal chunk is 320. With the full dataset, the accuracy measured is 22.4 %. For comparison, a random guess would have an accuracy of 8.3 %. However, state-of-the art